It's often said that Python comes with "batteries included", in reference to the vast library of modules and packages that are distributed with the interpreter.
Some important modules of the standard library:
In addition to the builtin numeric types in the Python standard library, there are several modules devoted to implementing other types and mathematical operations.
The math module defines logarithmic, exponentiation, trigonometric, and hyperbolic functions, as well as angular conversions and more. The cmath module implements similar functions, but can handle complex numbers.
Example:
In [1]:
import math
import cmath
# Complex
for cpx in [3j, 1.5 + 1j, -2 - 2j]:
# Polar coordinate conversion
plr = cmath.polar(cpx)
print ('Complex:', cpx)
print ('Polar:', plr, '(in radians)')
print ('Amplitude:', abs(cpx))
print ('Angle:', math.degrees(plr[1]), '(grades)')
The random module brings functions for random number generation.
Examples:
In [10]:
import random
import string
# Choose a letter
# print(string.printable)
# print(string.lowercase)
# print(help(random))
print (random.choice(string.ascii_uppercase))
print (random.choice([10, 200, 399, "Hello", "God", "Python", [222]]))
# Choose a number from 1 to 10
x = random.randrange(1, 100)
print (x)
# Choose a float from 0 to 1
print (random.random()*100)
In [3]:
lst = ["a", "b", "c", "e"]
dirs ={}
for l in lst:
dirs[l] = random.randrange(1, 100)
print(dirs)
In the standard library there is the decimal module that defines operations with real numbers with fixed precision.
Example:
In [4]:
from decimal import Decimal
t = 5.
for i in range(50):
t = t - 0.1
print ('Float:', t)
t = Decimal('5.')
for i in range(50):
t = t - Decimal('0.1')
print ('Decimal:', t)
With this module, it is possible to reduce the introduction of rounding errors arising from floating point arithmetic.
In version 2.6, the module fractions, which deals with rational numbers, is also available.
Example:
In [5]:
from fractions import Fraction
# Three fractions
f1 = Fraction('-2/3')
f2 = Fraction(3, 4)
f3 = Fraction('.25')
print ("Fraction('-2/3') =", f1)
print ("Fraction('3, 4') =", f2)
print ("Fraction('.25') =", f3)
# Sum
print (f1, '+', f2, '=', f1 + f2)
print (f2, '+', f3, '=', f2 + f3)
Fractions can be initialized in several ways: as a string, as a pair of integers, or as a real number. The module also has a function called gcd() which calculates the greatest common divisor (gcd) of two integers.
Files in Python are represented by objects of type *file*, which offer various methods for file operations. Files can be opened for reading ('r', which is the default), writing ('w'), or appending ('a'), in text or binary ('b') mode.
In Python:
The standard input, output and error are handled by Python as open files. The input in read mode and the other in the recording mode.
Sample of writing:
In [11]:
# The old way .... the C,C++ etc way
import sys
# Create an object of type file
temp = open('temp.txt', 'w')
# Write output
for i in range(20):
temp.write('%03d\n' % i)
temp.close()
temp = open('temp.txt')
temp1 = open('temp1.txt', "w")
# Write in terminal
for x in temp:
# writing in sys.stdout sends
# text to standard output
print("x")
sys.stdout.write(x)
temp1.write(x)
temp.close()
temp1.close()
In [12]:
# Python way
import sys
# Create an object of type file
with open('temp.txt', 'w') as temp:
# Write output
for i in range(20):
temp.write('%03d\n' % i)
with open('temp.txt') as temp:
# Write in terminal
for x in temp:
# writing in sys.stdout sends
# text to standard output
sys.stdout.write(x)
In [8]:
### TODO
# replace content of the file without temp file.
# on the fly content changes
txt = []
temp = open('temp.txt', 'r')
for t in temp.readlines():
# print("d")
txt.append(t.replace('0','11'))
with open('temp.txt', 'w') as tfile:
tfile.writelines(txt)
In [9]:
# print(help(open))
At each iteration in the second loop, the object returns a line from the file each time.
Reading example:
In [13]:
import sys
import os.path
fn = 'temp.txt'
if os.path.exists(fn):
# Numbering lines
for i, s in enumerate(open(fn)):
print (i + 1, s),
It is possible to read all the lines with the method readlines():
In [11]:
# Prints a list with all the lines from a file
print (open('temp.txt').readlines())
In [14]:
print(len(open('temp.txt').readlines())) # 20
print(len(open('temp.txt').read())) # 4x20
Modern operating systems store files in hierarchical structures called file systems.
Several features related to file systems are implemented in the module os.path, such as:
os.path.basename(): returns the final component of a path.os.path.dirname(): returns a path without the final component.os.path.exists(): returns True if the path exists or False otherwise.os.path.getsize(): returns the size of the file in bytes.glob is another module related to the file system:
In [13]:
# print(help(os.path))
In [21]:
import os.path
a = r"c:\users\mayank"
b = r"warzone\myprogram.conf"
c = os.path.join(a, b)
print(c)
print(os.path.split(c))
print(os.path.split(c)[1])
print(os.path.splitext(c))
print(os.path.splitdrive(c))
In [15]:
import os.path
import glob
# Shows a list of file names
# and their respective sizes
for arq in sorted(glob.glob('*.*')):
print (arq, os.path.getsize(arq))
The glob.glob() function returns a list of filenames that meet the criteria passed as a parameter in a similar way to the ls command available on UNIX systems.
The module os implements some functions to facilitate the creation of temporary files, freeing the developer from some concerns, such as:
Example:
In [22]:
import tempfile
# create a temporary file and write some data to it
fp = tempfile.TemporaryFile()
fp.write(b'Hello world!')
# read data from file
fp.seek(0)
fp.read()
# close the file, it will be removed
fp.close()
# create a temporary file using a context manager
with tempfile.TemporaryFile() as fp:
fp.write(b'Hello world!')
fp.seek(0)
fp.read()
# file is now closed and removed
# create a temporary directory using the context manager
with tempfile.TemporaryDirectory() as tmpdirname:
print('created temporary directory', tmpdirname)
# directory and contents have been removed
The objects of type file also have the method seek(), which allow going to any position in the file.
In [17]:
# print(help(type(fp)))
There is also the tempnam() function, which returns a valid name for temporary file, including a path that respects the conventions of the operating system. However, it is up to the developer to ensure that the routine is used so as not to compromise the security of the application.
Python has modules to work with multiple formats of compressed files.
Example of writing a ".zip" file:
In [23]:
"""
Writing text in a compressed file
"""
import zipfile
text = """
**************************************
This text will be compressed and ...
... stored inside a zip file.
***************************************
"""
# Creates a new zip
zip = zipfile.ZipFile('new_zip.zip', 'w',
zipfile.ZIP_DEFLATED)
# Writes a string in zip as if it were a file
zip.writestr('text.txt', text)
zip.writestr('text1.txt', text)
zip.writestr('text2.txt', text)
# closes the zip
zip.close()
Reading example:
In [19]:
"""
Reading a compressed file
"""
import zipfile
# Open the zip file for reading
zip = zipfile.ZipFile('arq1.zip')
# Gets a list of compressed files
arqs = zip.namelist()
for arq in arqs:
# Shows the file name
print ('File:', arq)
# get file info
zipinfo = zip.getinfo(arq)
print ('Original size:', zipinfo.file_size)
print ('Compressed size:', zipinfo.compress_size)
# Shows file content
print (zip.read(arq))
Python also provides modules for gzip, bzip2 and tar formats that are widely used in UNIX environments.
In the standard library, Python also provides a module to simplify the processing of files in CSV (Comma Separated Values) format.
In CSV format, the data is stored in text form, separated by commas, one record per line.
Writing example:
In [25]:
import csv
with open('eggs.csv', 'w', newline='') as csvfile:
spamwriter = csv.writer(csvfile, delimiter=',',
quotechar='|', quoting=csv.QUOTE_MINIMAL)
spamwriter.writerow(['Spam'] * 5 + ['Baked Beans'])
spamwriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam'])
In [26]:
lst = [["a", "a1"], ["b", "b1"], ["d", "d1 ,one"], "c", "f" ]
with open('list.csv', 'w', newline='') as csvfile:
listWriter = csv.writer(csvfile)
for l in lst:
listWriter.writerow(l)
Reading example:
In [27]:
import csv
with open("eggs.csv", newline='') as csvfile:
for row in csv.reader(csvfile, delimiter=' ', quotechar='|'):
print(', '.join(row))
In [23]:
import os
import sys
import platform
def uid():
"""
uid() -> returns the current user identification
or None if not possible to identify
"""
# Ambient variables for each operating system
us = {'Windows': 'USERNAME',
'Linux': 'USER'}
u = us.get(platform.system())
return os.environ.get(u)
print ('User:', uid())
print ('plataform:', platform.platform())
print ('Current dir:', os.path.abspath(os.curdir))
exep, exef = os.path.split(sys.executable)
print ('Executable:', exef)
print ('Executable dir:', exep)
Process execution example:
In [24]:
###################################
### TODO ##########################
###################################
###################################
###################################
# import sys
# from subprocess import Popen, PIPE
# # ping
# cmd = 'ping -c 1 '
# # No Windows
# if sys.platform == 'win32':
# cmd = 'ping -n 1 '
# # Local just for testing
# host = '127.0.0.1'
# # Comunicates with another process
# # a pipe with the command stdout
# py = Popen(cmd + host, stdout=PIPE)
# # Shows command output
# print (py.stdout.read())
The subprocess module provides a generic way of running processes with Popen() function which allows communication with the process through operating system pipes.
Python has two modules to handle time:
Example with time:
In [29]:
import time
# localtime() Returns a date and local time in the form
# of a structure called struct_time, which is a
# collection with the items: year, month, day, hour, minute,
# secund, day of the week, day of the year and e daylight saving time
print (time.localtime())
# asctime() returns a date and hour with string, according to
# operating system configuration
print (time.asctime())
# time() returns system time in seconds
ts1 = time.time()
# gmtime() converts seconds to struct_time
tt1 = time.gmtime(ts1)
print (ts1, '->', tt1)
# Adding an hour
tt2 = time.gmtime(ts1 + 3600)
# mktime() converts struct_time to seconds
ts2 = time.mktime(tt2)
print (ts2, '->', tt2)
# clock() returs time since the program started, in seconds
print ('The program took', time.clock(), \
'seconds up to now...')
# Counting seconds...
for i in range(5):
# sleep() waits the number of seconds specified as parameter
time.sleep(1)
print (i + 1, 'second(s)')
In [28]:
def countdown(x):
# Counting seconds...
for i in range(x):
# sleep() waits the number of seconds specified as parameter
time.sleep(1)
print (i + 1, 'second(s)')
countdown(4)
print("countdown completed")
In datetime, four types are defined for representing time:
Example:
In [27]:
import datetime
# datetime() receives as parameter:
# year, month, day, hour, minute, second and
# returns an object of type datetime
dt = datetime.datetime(2020, 12, 31, 23, 59, 59)
# Objects date and time can be created from
# a datetime object
date = dt.date()
hour = dt.time()
# How many time to 12/31/2020
dd = dt - dt.today()
print ('Date:', date)
print ('Hour:', hour)
print ('How many time to 12/31/2020:', dd)
In [ ]:
The textwrap module provides some convenience functions, as well as TextWrapper, the class that does all the work. If you’re just wrapping or filling one or two text strings, the convenience functions should be good enough; otherwise, you should use an instance of TextWrapper for efficiency
In [28]:
import textwrap
txt = " The `textwrap` module provides some convenience functions, as well as `TextWrapper`, the class that does all the work. If you’re just wrapping or filling one or two text strings, the convenience functions should be good enough; otherwise, you should use an instance of `TextWrapper` for efficiency"
print(txt)
for t in textwrap.wrap(txt):
print(t)
for t in textwrap.wrap(txt, width=100):
print(t)
for t in textwrap.wrap(txt, width=60, initial_indent="* "):
print(t)
In [29]:
print(textwrap.fill(txt))
In [30]:
print(textwrap.shorten(txt, width=60))
print(textwrap.shorten(txt, width=50))
s = textwrap.shorten(txt, width=25, placeholder="...")
print(s)
print(len(s))
s = textwrap.shorten(txt, width=20, placeholder="...")
print(s)
print(len(s))
In [31]:
textwrap.dedent(txt)
Out[31]:
In [32]:
print(textwrap.indent(txt.strip().replace(",", '\n'), '$ '))
Write a Python program to read an entire text file.
Write a Python program to read first n lines of a file.
Write a Python program to append text to a file and display the text.
Write a Python program to read last n lines of a file.
Write a Python program to read a file line by line and store it into a list.
Write a Python program to read a file line by line store it into a variable.
Write a Python program to read a file line by line store it into an array.
Write a python program to find the longest words.
Write a Python program to count the number of lines in a text file.
Write a Python program to count the frequency of words in a file.
Write a Python program to get the file size of a plain file.
Write a Python program to write a list to a file.
Write a Python program to copy the contents of a file to another file .
Write a Python program to combine each line from first file with the corresponding line in second file.
Write a Python program to read a random line from a file.
Write a Python program to assess if a file is closed or not.
Write a Python program to remove newline characters from a file.
csv module